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DETAILED ACTION 

Status of Claims 

1 . This action is in reply to the amendment filed on 6/22/2009. 

2. Claims 29-33, 38, 43-45, 49-51 have been amended. 

3. Claims 34-37, 39-42, 46-48 and 52-56 have been canceled. 

4. Claims 29-33, 38, 43-45 and 49-51 are currently pending and have been examined. 

Response to Applicant's Arguments 

5. Applicant's amendments and arguments filed on 6/22/2009 have been fully considered, but they 
are not persuasive and discussed in the next section. Applicant is reminded that the claims must be 
given its broadest, reasonable interpretation. 

6. With regard to claims 29, 43, 49: 

• Applicant argues that "Nowhere does Abe disclose, teach or suggest a randomization of actions to 
allow for explorations of other actions in a RL algorithm as does the present invention". 

Examiner respectfully disagrees. Abe in at least paragraph 90 discloses the epsilon. -greedy method 
which employs the state space by employing a stochastic policy that chooses an action at random 
with probability. With reference to Wikipedia.org, please see "Multi-armed_bandit" for Multi for 
Epsilon-greedy strategy means. Additionally, as per Applicant's specification, "all the existing RL 
algorithm are variants of the above basic procedure (paragraph 90)" and "FIG. 4 is a flowchart 
depicting the reinforcement learning algorithm, as it exists in the art ( paragraph 84)". 

• Applicant argues that Abe does not disclose, teach or suggest at least the present invention's 
features of: " recommending, by said computer, a set of possible marketing strategies to determine an 
optimal marketing strategy by using a modified RL algorithm". 

Examiner respectfully disagrees. Abe in at least paragraphs 13 and 14 discloses a modified 
reinforcing learning algorithm. Abe in at least paragraph 19 discloses an inventive method that is 
applied to cross -channel optimized marketing by providing customer data including stimulus- 
response history data from a plurality of channels, integrating the customer data and automatically 
generating channel specific actionable rules based on customer data. For instance, in this case the 
method may optimize cross- channel cumulative profits. Abe also in paragraph 21 discloses 
automatically generating actionable rules that includes transforming an output of value function 
estimation into a set of actionable rules. Specifically, transforming may include a transformation 
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model of the value function into a set of actionable rules, so as to output a set of if-then style rules 
that specify what action to take given a set of features values corresponding to a customer, and to 
specify an action that corresponds to the action with an approximately maximum value according to 
the value function for a given set of feature values. Furthermore, Abe in at least paragraph 90 
discloses the epsilon.-greedy method which employs the state space by employing a stochastic policy 
that chooses an action at random with probability. 
• Applicant argues that Mittal does not 

a. disclose, teach or suggest a randomization of actions to allow for explorations of other 
actions in a RL algorithm as does the present invention". 

b. disclose, teach or suggest recommending, by said computer, a set of possible marketing 
strategies to determine an optimal marketing strategy by using a modified RL algorithm". 

Examiner respectfully disagrees. Mittal reference was used to address the limitation of: inputting, to a 
computer by a merchant, said at least one specified merchant objective, said at least one specified 
merchant objective including said specified time period and said specified budget as constraints. 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 

rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art 
are such that the subject matter as a whole would have been obvious at the time the invention was made to 
a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be 
negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), 
that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are 
summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness or 
nonobviousness. 

8. Claims 29-33, 38, 43-45, 49, 50 and 51 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Abe et al, US Pub No. 2004/0015386 in view of Mittal et al, US Pub No: 2004/01 17239 
A1. 

Claims 29, 43 and 49: 

Abe discloses: 
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• contacting said merchant by said customer, said customer having a customer 
profile stored in said computer, said customer profile including a history of 
interactions between said customer and said merchant, each of said 
interactions corresponding to a state of said customer (see at least 
paragraphs 11-14, 67, 152-157 and 167); 

• generating, by said computer, possible marketing strategies, each of said 
possible marketing strategies comprising initiatives, which are deployed 
together in a given sequence each of said initiatives being associated with a 
marketing channel and being deployed for a variable time; 
wherein said initiatives include any of bundling of products, cross-sells, up- 
sells, coupons, discounts, promotions, advertisements, surveys, and 
customer feedback; 

See at least paragraphs 11-14, 179-181 and 186; 

• recommending, by said computer, a set of possible marketing strategies, 
along with a deployment probability of each of said set of possible marketing 
strategies to determine an optimal marketing strategy by using a modified 
Reinforcement Learning (RL) algorithm (see at least paragraphs 90-93); 

• said contacting said merchant by said customer corresponds to a perception 
of said state based on said customer profile (see at least paragraphs 14, 19, 
67, 167-170); 

• said initiatives deployed by any of said possible marketing strategies 
correspond to actions (see at least paragraphs 14, 19, 67, 129 and 152) ; 

• a total expected reward is estimated for all states and all actions 
corresponding to said possible marketing strategies, and estimating said total 
expected reward for all states and all actions comprises (see at least 
paragraphs 15,16, 31 and 79); 

• for every state, identifying a specific action that gives a maximal value for a 
corresponding total expected reward; selecting another action corresponding 
to a randomization of deployment for said specific action (see at least 
paragraphs 67, 74-79 and 150); 

• recording a total expected reward for execution of said another action and a 
resultant state resulting from the action of said another action: updating an 
estimated value of said total expected reward with the said total expected 
reward for execution of said another action and said resultant state, 
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according to a discount factor (see at least paragraphs 79, 123, 150 and 
153); 

• iteratively repeating said identifying a specific action, said selecting another 
action, said recording a total expected reward, and said updating an 
estimated value of said total expected reward, to determine a best value for 
said total expected reward that identifies said optimal marketing strategy (see 
at least paragraphs 14, 74-79 and 153-157); 

• deploying, by said computer, said optimal market strategy to said customer; 
recording, by said computer, a response of said customer to said optimal 
market strategy in an updated customer profile (see at least paragraphs 
152,171 and 176); 

• iteratively repeating, by said computer, said recommending a set of said 
possible marketing strategies along with a deployment probability of each of 
said set of said possible marketing strategies, said deploying said optimal 
market strategy, and said recording a response of said customer, while said 
time period, specified by said merchant, has not expired, otherwise stopping 
(see at least paragraphs 65, 67 and 152-156); 

Abe does not specifically disclose, but Mittal, however, discloses: 

• inputting to a computer, a merchant objective that is specified by a merchant, 
said merchant objective being associated with a time period (see at least 
paragraph 15); 

The combination of Abe/ Mital discloses: 

• a total time of each of said possible marketing strategies being equal to a 
sum of all of said initiatives deployed, 

• wherein said generating of said possible marketing strategies is subject to 
constraints of cost and a specified total time of each of said possible 
marketing strategies; 

Abe in at least paragraphs 12 and 122 discloses an invention of sequential decision 
making for customer relationship management on reinforcement learning framework 
to optimize a sequence of cost-sensitive decisions in order to maximize the total 
benefits occurred over time. Abe also in at least paragraph 122 discloses a time 
window that is used to summarize recent promotion -response behaviors, where only 
the last 16 of 22 campaigns were used for episode data. Furthermore, Abe in at 
least paragraph 185 and fig (18 and 19) with the associated text discloses an 
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analytics for customer relationship management for optimizing targeted marketing 
across all channels for lifetime value maximization. 

Mittal in at least paragraph 15 discloses any marketing research experiment has an 
associated budget, one or more target user segments and a time period for 
completion, the merchant desires to complete the experiment within the specified 
budget and within the defined time period to obtain the required information form one 
or more set of target users/customers. Mittal also in at least paragraph 46 discloses 
a mechanism for minimizing the cost and the total time required to complete the 
experiment. Furthermore, Mittal in at least paragraph 50 discloses flowchart 
specification of an experiment parameter by the merchant. The parameters comprise 
the time period for the experiment and the experiment budget. 
It would have been obvious to one of ordinary skill in the art at the time of the 
invention to include in the reinforcement learning algorithm marketing strategy of Abe 
the ability to specify total time of each marketing strategy and budget/time 
constraints as taught by Mittal since the claimed invention is merely a combination of 
old elements, and in the combination each element merely would have performed the 
same function as it did separately, and one of ordinary skill in the art would have 
recognized that the results of (of providing the merchants with an optimal marketing 
strategy developed within a set budget and time period) the combination were 
predictable. 

Claims 30, 44 and 50: 

Abe/ Mittal disclose the limitations as shown above. 
Abe further discloses: 

• selecting at least one initiative that enables an addressing of said objective; 

• determining a sequence in which selected initiatives can be deployed, if more 

than one initiative is selected; 

• combining the selected initiatives in the determined sequences to obtain said 

possible marketing strategies; 
See at least paragraphs 70, 176 and 199; 

Claims 31 and 32: 

Abe/ Mittal disclose the limitations as shown above. 
Abe further discloses: 
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• varying parameters of said selected initiatives to generate new initiatives 
corresponding to new states during said specified time period (see at least 
paragraphs 195-199); 

• varying deployment time of said selected initiatives (see at least paragraphs 201 
and 202); 



Claims 33, 38, 45 and 51: 

Abe/ Mittal disclose the limitations as shown above. 
Abe further discloses: 

• wherein determining an optimal marketing strategy further comprises constraints 
corresponding to marketing channels 

• assigning customer's preferences for marketing channels, cost and effectiveness of 
different marketing channels, and the specified budget as effective constraints; 

See at least paragraphs 185.192-196 and fig 19 with the associated text; 



Conclusion 



9. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth 
in 37 CFR 1.136(a). 

10. A shortened statutory period for reply to this final action is set to expire THREE MONTHS from 
the mailing date of this action. In the event a first reply is files within TWO MONTHS from the mailing date 
of this final action and the advisory action is not mailed until after the end of the THREE-MONTH 
shortened statutory period, then the shortened statutory period will expire on the date the advisory action 
is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later than SIX Months 
from the mailing date of this final. 

11. Any inquiry concerning this communication or earlier communications from the examiner should 
be directed to Affaf Ahmed whose telephone number is 571-270-1835. The examiner can normally be 
reached on Monday - Friday, 8:30 am-6:00 pm est, alt Fridays off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Eric 
Stamber can be reached at 571-272-6724. The fax phone number for the organization where this 
application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent Application 
Retrieval (PAIR) system. Status information for published applications may be obtained from either 
Private PAIR or Public PAIR. Status information for unpublished applications is available through Private 
PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 
866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or 
access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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